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REFERENCE TO RELATED APPLICATIONS 

[000 

1.] This application is based on US Provisional Patent Application No. 

60/228,963 filed on August 29, 2000. 

FIELD OF THE INVENTION 
[000 2.] The present invention relates to electronic archiving, indexing 

and retrieving images. 

BACKGROUND OF THE INVENTION 
[000 3. J The following U.S. Patents are believed to represent the current 

state of the art: 6,263,113 Bl; 6,108,437; 6,035,074; 6,006,231; 5,978,507; 5,963,670; 
5,915,038; 5,905,807; 5,896,462; 5,793,888; 5,764,790. 

SUMMARY OF THE INVENTION 
[000 4.] The present invention seeks to provide an improved system and 

methodology for archiving, indexing and retrieving images. 

[000 5.] There is thus provided in accordance with a preferred 

embodiment of the present invention a method for indexing images of persons. The 
method includes providing image data relating to the images of persons to a suitably 
programmed computer, employing a first suitably programmed computer functionality 
to provide face recognition of the images, employing a second suitably programmed 
computer functionality to group the images according to faces recognized therein and 
employing a third suitably programmed computer functionality to provide an index of 
groups of the images organized according to faces recognized therein. 
[000 6.] There is also provided in accordance with a preferred 

embodiment of the present invention a system for indexing images of persons. The 
system includes an image data source providing image data relating to the images of 
persons to a suitably programmed computer, a first suitably programmed computer 
functionality providing face recognition of the images, a second suitably programmed 
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computer functionality grouping the images according to faces recognized therein and a 
third suitably programmed computer functionality providing an index of groups of the 
images organized according to faces recognized therein. 

[000 7.] Further in accordance with a preferred embodiment of the present 

invention the method also includes employing the index for retrieving, viewing and 
optionally downloading images of an individual person. 

[000 8.] Still further in accordance with a preferred embodiment of the 

present invention at least two of the first, second and third suitably programmed 
computer functionalities are performed by the suitably programmed computer. 
[000 9.] Additionally or alternatively, all of the first, second and third 

suitably programmed computer functionalities are performed by the suitably 
programmed computer. 

[00 10.] Moreover in accordance with a preferred embodiment of the 

present invention the index employs a recognized face as an index icon. 

[00 11.] Preferably, the steps of providing image data and retrieving, 

viewing and optionally downloading are carried out via the Internet. 

[00 12.] Further in accordance with a preferred embodiment of the present 

invention the step of providing image data employs a film camera and a scanner. 

[0° 13 «] Alternatively, the step of providing image data employs a digital 

camera. 

[00 14.] Still further in accordance with a preferred embodiment of the 

present invention the index employs a recognized face as an index icon. 
[00 15.] There is also provided in accordance with another preferred 

embodiment of the present invention a method for classifying images of persons in 
photographs. The method includes the steps of photographing a plurality of persons in a 
plurality of scenes, wherein not all of the persons appear in all of the scenes, to provide 
a plurality of photographs, computer analyzing the plurality of photographs to detect at 
least the faces of persons in each of the scenes and automatically grouping the 
photographs according to at least the faces of the persons appearing therein. 
[00 16.] There is further provided in accordance with yet another 

preferred embodiment of the present invention a system for classifying images of 
persons in photographs. The system includes a photography subsystem photographing a 
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plurality of persons in a plurality of scenes, wherein not all of the persons appear in all 
of the scenes, to provide a plurality of photographs and a computer analysis and 
photograph grouping subsystem for computer analyzing the plurality of photographs to 
detect at least the faces of persons in each of the scenes and automatically grouping the 
photographs according to at least the faces of the persons appearing therein. 
[00 17.] Further in accordance with a preferred embodiment of the present 

invention the method for classifying images also includes indexing the plurality of 
photographs at least partially in accordance with the faces of the persons appearing 
therein. 

[00 18.] Still further in accordance with a preferred embodiment of the 

present invention the step of photographing includes photographing the plurality of 
persons while they are bearing unique identification indications. The step of computer 
analyzing includes face recognition of the faces of the persons appearing in the 
photographs, recognition of the unique identification indications and correlation of the 
faces with the unique identification indications. 

[00 19.] Additionally in accordance with a preferred embodiment of the 

present invention the step of computer analyzing includes image indication assisted face 
recognition. 

[00 20.] Further in accordance with a preferred embodiment of the present 

invention the step of face recognition includes image indication assisted face 
recognition. 

[00 21.] Moreover in accordance with a preferred embodiment of the 

present invention the step of grouping the photographs employs the step of correlating 
the faces with the unique identification indications. 

[00 22.] Additionally in accordance with a preferred embodiment of the 

present invention the steps of grouping and indexing the photographs employs 
correlating the faces with the unique identification indications. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[00 23.] The present invention will be understood and appreciated more 

fully from the following detailed description, taken in conjunction with the drawings in 
which: 
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[00 24.] Figs. 1A and IB are simplified pictorial illustrations of a system 

and methodology for archiving images in accordance with two variations of a preferred 
embodiment of the present invention; 

[00 25.] Figs. 2A and 2B are simplified pictorial illustrations of a system 

and methodology for archiving images in accordance with two variations of another 
preferred embodiment of the present invention; 

[00 26.] Fig. 3 is a simplified flowchart illustrating the functionality of the 

embodiment of Figs. 1A and IB; 

[00 27.] Fig. 4 is a simplified flowchart illustrating the functionality of the 

embodiment of Figs. 2A and 2B; 

[00 28.] Fig. 5 is a simplified flowchart illustrating photograph grouping 

and indexing functionality employed in accordance with a preferred embodiment of the 
present invention; and 

[00 29.) Fig. 6 is a simplified flowchart illustrating image indication 

assisted face recognition functionality employed in accordance with a preferred 
embodiment of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
[00 30.] Reference is now made to Figs. 1 A and IB, which are simplified 

pictorial illustrations of a system and methodology for archiving images in accordance 
with two variations of a preferred embodiment of the present invention. 
[00 31.] As seen in Fig. 1 A, scenes in which various persons are present 

are photographed in a plurality of settings and groupings. Not all of the persons appear 
in each scene. The photographs are provided, typically in electronic form, from a 
camera 10 to a suitably programmed computer 12, which is operative to index the 
photographs according to images of persons appearing therein. Transmission of image 
data to the computer 12 may be effected directly or via a computer network, such as the 
Internet, in any suitable manner. Thus, the computer 12 is operative typically to provide 
a thumbnail image 14 of each person photographed, which thumbnail may serve as an 
index icon for viewing the thumbnail images of all of the photographs 16 in which that 
person appears. 
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[00 32.] In accordance with a preferred embodiment of the present 

invention, the computer 12 carries out face recognition on the images of the persons in 
the various photographs and is operative to recognize the same people in different 
photographs notwithstanding certain changes in their visual appearance and orientation. 
[00 33.] Turning to Fig. IB it is seen that the scenes, in which various 

persons are present, are similarly photographed in a plurality of settings and groupings 
wherein not all of the persons appear in each scene. Either or both a conventional film 
camera 20 and a digital camera 22 may be employed. Where a conventional film 
camera is employed, the photographs or negatives 24 may be scanned by a scanner 26 
in a conventional manner and the electronic output thereof may be provided to a 
suitably programmed computer 32. The output from digital camera 22 may be provided 
in electronic form to computer 32. Transmission of image data to the computer 32 may 
be effected directly or via a computer network 28, such as the Internet, in any suitable 
manner. Thus, the computer 32 is operative typically to provide a thumbnail image 34 
of each person photographed, which thumbnail may serve as an index icon for viewing 
the thumbnail images of all of the photographs 36 in which that person appears. 
[00 34.] As in the embodiment of Fig. 1A, computer 32 preferably carries 

out face recognition on the images of the people in the various photographs and is 
operative to recognize the same people in different photographs notwithstanding certain 
changes in their visual appearance and orientation. 

[00 35.] It is to be appreciated that the present invention does not require 

that the computer 32 be located adjacent to either the scanner 26 or to the digital camera 
22. A preferred embodiment of the invention employs communication over a computer 
network 28, such as the Internet, with a remote processing computer, such as computer 
32, and retrieving and viewing of the index of the icon images 34 and retrieving and 
viewing groups of thumbnail images 36 for each icon and optionally downloading the 
complete photograph of one or more of the thumbnail images 36 from the computer 32, 
over the Internet to a plurality of users. This preferred Internet embodiment is also 
applicable to the embodiment of Fig. 1A. 

[00 36.] Reference is now made to Figs. 2A and 2B, which are simplified 

pictorial illustrations of a system and methodology for archiving images in accordance 
with two variations of another preferred embodiment of the present invention. 
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[00 37.] As seen in Fig. 2A, similarly to that shown in Figs. 1A and IB, 

scenes in which various persons are present are photographed in a plurality of settings 
and groupings wherein not all of the persons appear in each scene. 
[00 38.] In distinction to that shown in Figs. 1A and IB, each person in 

each scene preferably wears an identification indication 50 on the upper part of his 
body. This identification indication is preferably a multi-color, color coded 
identification indication commercially available from ImagelD Inc. of One Perm Plaza, 
New York, New York under the VIVIDOT trademark. A preferred embodiment of the 
identification indication is described in applicant/assignee's U.S. Patent Application 
Serial No. 09/508,300, filed March 9, 2000 and entitled "Image Identification and 
Delivery System", the disclosure of which is hereby incorporated by reference and 
appears in Published PCT Patent Application WO 00/0471 1, the disclosure of which is 
hereby incorporated by reference. The identification indication 50 worn by each person 
is unique and visually distinguishable from the identification indication 50 worn by 
each other person. 

[00 39.] As in the embodiments of Figs. 1A and IB, the photographs are 

provided, typically in electronic form, from a camera 51 to a suitably programmed 
computer 52, which is operative to group the photographs according to each person 
appearing therein. In this embodiment, each person is distinguished from each other 
person, not only by the face of the person but also by the identification indication 50 
worn by that person. The computer reads the identification indication 50 and typically 
derives a code sequence uniquely corresponding thereto. Similarly, the computer 
preferably employs a code sequence uniquely corresponding to each face in each 
photograph. Thus, it is appreciated that in the embodiment of Figs. 2A and 2B, the 
computer preferably employs image data taken from the photographs to key each face 
to a unique identification indication 50 and vice versa. 

[00 40.] Thus, the computer 52 is operative, preferably by using not only 

face recognition but also recognition of the unique identification indications 50, to 
provide a thumbnail image 54 of each person photographed, which thumbnail may 
serve as an icon for viewing the thumbnail images of all of the photographs 56 in which 
that person appears. Transmission of image data to the computer 52 may be effected 
directly or via a computer network, such as the Internet, in any suitable manner. 
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[00 41.] By virtue of the face recognition carried out by the computer 52 

on the images of the people in the various photographs and by virtue of the recognition 
of the unique identification indications, the computer is provided with an enhanced 
ability to recognize the same people in different photographs notwithstanding certain 
changes in their visual appearance and orientation. Thus, even if only one of the face 
and the identification indication is recognizable in a photograph, the photograph can 
nevertheless be indexed. 

[00 42.] It is also a particular feature of the embodiment of Fig. 2A that 

the suitably programmed computer 52 carries out face recognition with the assistance of 
the identification indication 50. Preferably, the computer 52 recognizes each 
identification indication 50 and identifies its location as the upper part of a human body. 
The computer 52 then looks for a face at a location generally above and adjacent to the 
upper part of the human body. This identification indication aided face recognition 
enables enhanced ease, speed and accuracy of face recognition. Thus, the computer 52 
is operative typically to provide an index icon 54 for viewing all of the groups of 
thumbnail images 56 in which that person appears. The viewer may optionally 
download the complete photograph of a thumbnail image by double clicking on the 
thumbnail image 56. 

[00 43.] Turning to Fig. 2B, it is seen that the scenes in which various 

persons are present are similarly photographed in a plurality of settings and groupings 
wherein not all of the persons appear in each scene. Either or both a conventional film 
camera 70 and a digital camera 72 may be employed. Where a conventional film 
camera is employed, the photographs or negatives 74 may be scanned by a scanner 76 
in a conventional manner and the electronic output thereof may be provided to a 
suitably programmed computer 82. The output from digital camera 72 may be provided 
in electronic form to computer 82. Transmission of image data to the computer 82 may 
be effected directly or via a computer network 88, such as the Internet, in any suitable 
manner. 

[00 44.] As in the embodiment of Fig. 2A, computer 82 carries out image 

indication assisted face recognition on the images of the people in the various 
photographs and is operative to recognize the same people in different photographs 
notwithstanding certain changes in their visual appearance and orientation. 
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[00 45.] It is to be appreciated that the present invention does not require 

that the computer 82 be located adjacent to either the scanner 76 or to the digital camera 
72. A preferred embodiment of the invention employs communication over a computer 
network 88, such as the Internet, with a remote processing computer, such as computer 
82, and retrieving and viewing of the index icons 84. The user may then retrieve and 
view groups of the thumbnail images 86 for each icon and optionally download a 
complete photograph of one or more of the thumbnail images 86 from the computer 82, 
over the Internet, to a plurality of users. This preferred Internet embodiment is also 
applicable to the embodiment of Fig. 2 A. 

[00 46.] Reference is now made to Fig. 3, which is a simplified flowchart 

illustrating the functionality of the embodiment of Figs. 1A and IB. As seen in Fig. 3, a 
plurality of scenes are photographed. Each scene typically includes one or more 
persons. Typically, each person photographed in a scene does not appear in all of the 
photographs. 

[00 47.] If a film camera is employed, the film is developed and may be 

printed. Either the negatives or the prints are scanned using a conventional scanner. If a 
digital camera is employed, the electronic output thereof is downloaded from the 
camera using any suitable technique or media. 

[00 48.] The resulting image content in electronic form may be supplied 

to a suitably programmed computer in any suitable manner. The computer may be any 
suitable computer. In accordance with a preferred embodiment of the present invention, 
the image content may be transmitted to a remote computer via the Internet. 
[00 49.] The suitably programmed computer preferably is programmed 

with face recognition software. Such software may be entirely conventional. Examples 
of commercially available software for this purpose include: 

[00 50.] FACEIT R software, commercially available from Visionics 

Corporation of 5600 Rowland Road, Minnetonka, MN 55343 U.S.A. and 
[00 51.] TrueFace software, commercially available from eTrue, Inc. of 

144 Turnpike Road, Suite 100, Southboro, MA 01772 U.S.A. 

[00 52.] Upon achievement of face recognition of faces of persons in the 

various images represented by the image content, the same computer or another suitably 
programmed computer is employed to group photographs according to the persons 
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whose faces are recognized therein. Each such group of photographs is preferably 
indexed. A preferred image icon is a thumbnail image of the face of each person 
appearing in one or more photographs. 

[00 53.] Preferred functionality for carrying out the face recognition- 

based grouping and indexing functionalities described in the preceding paragraph is 
described hereinbelow with reference to Fig. 5. 

[00 54.] Once an index of photographs organized by the face recognized 

therein has been generated, it may be retrieved and viewed by any authorized user. The 
group of thumbnail images associated with each index icon may be retrieved and 
viewed by any authorized user by preferably clicking on an index icon, which may 
advantageously be a thumbnail image of a recognized face. 

[00 55.] Reference is now made to Fig. 4, which is a simplified flowchart 

illustrating the functionality of the embodiment of Figs. 2A and 2B. As seen in Fig. 4, a 
plurality of scenes are photographed. Each scene typically includes one or more 
persons. Typically, each person photographed in a scene does not appear in all of the 
photographs. 

[00 56.] In distinction to that described hereinabove with reference to Fig. 

3, each person in each scene preferably wears a unique identification indication on his 
body. If a film camera is employed, the film is developed and may be printed. Either the 
negatives or the prints are scanned using a conventional scanner. If a digital camera is 
employed, the electronic output thereof is downloaded from the camera using any 
suitable technique or media. 

[00 57.] The resulting image content in electronic form may be supplied 

to a suitably programmed computer in any suitable manner. The computer may be any 
suitable computer. In accordance with a preferred embodiment of the present invention, 
the image content may be transmitted to a remote computer via the Internet. 
[00 58.] The suitably programmed computer preferably is programmed 

with identification indication recognition software, which is available for license on 
commercial terms from ImagelD Inc. Once an identification indication has been 
recognized, the suitably programmed computer employs the location of the 
identification indication as a directional cue to assist in face recognition. 
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[00 59.] Preferred functionality for carrying out image indication assisted 

face recognition is described hereinbelow with reference to Fig. 6. 

[00 60.] Preferably once both a face and a corresponding unique 

identification indication have been recognized, they are keyed together and both 
employed for identifying a given person in a photograph. 

[00 61.] Upon achievement of face recognition of faces and unique 

identification indications of persons in the various images represented by the image 
content, the same computer or another suitably programmed computer is employed to 
group photographs according to the persons at least one of whose faces and 
identification images are recognized therein. Each such group of photographs is 
preferably indexed. A preferred image icon is a thumbnail image of the face of each 
person appearing in one or more photographs. 

[00 62.] Preferred functionality for carrying out the face recognition- 

based grouping and indexing functionalities described in the preceding paragraph is 
described hereinbelow with reference to Fig. 5. 

[00 63.] Once an index of photographs organized by the face recognized 

therein has been generated, it may be retrieved and viewed by any authorized user. The 
group of thumbnail images associated with each index icon may be retrieved and 
viewed by any authorized user by preferably clicking on an index icon for viewing all 
of the groups of the thumbnail images in which that person appears. The viewer may 
optionally download a complete picture of a thumbnail image by double clicking on the 
thumbnail image. 

[00 64.] Reference is now made to Fig. 5, which is a simplified flowchart 

illustrating photograph grouping and indexing functionality employed in accordance 
with a preferred embodiment of the present invention. 

[00 65.] As seen in Fig. 5, face detection is preferably performed on all 

faces appearing in a plurality of photographs to be grouped, using any suitable face 
detection software, such as, for example: 

[00 66.] FACEIT R software, commercially available from Visionics 

Corporation of 5600 Rowland Road, Minnetonka, MN 55343 U.S.A. and 
[00 67.] TrueFace software, commercially available from eTrue, Inc. of 

144 Turnpike Road, Suite 100, Southboro, MA 01772 U.S.A. 
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[00 68.] Preferably, each detected face is compared with all other detected 

faces in order to determine which detected faces match each other to at least a 
predetermined extent. Additionally a similar functionality is preferably carried out with 
recognized identification indications in order to determine which detected identification 
indications match each other. Photographs each containing an image of what appears to 
be the same face and or an image of an identification indication corresponding to that 
face are grouped together. It is appreciated that within each group, the various images of 
the "same face" may resemble each other to a greater or lesser extent and in some cases, 
images of the "same face" may not resemble each other. It is preferably a condition for 
a photograph belonging to a group that the image of a face in the photograph resembles 
most of the images of the face in the other photographs in the group to at least a given 
extent. 

[00 69.] An image index icon is selected to be a representative face image 

for each group. Typically it is the face image which has the highest extent of matching 
to the other images of the same face in the various photographs in the group. The image 
index icon is displayed to a user to represent to the user, the face which is common to 
each group and thus to indicate the identity of the person who is pictured in each of the 
photographs in the group. 

[00 70.] Reference is now made to Fig. 6, which is a simplified flowchart 

illustrating image indication assisted face recognition functionality employed in 
accordance with a preferred embodiment of the present invention. As seen in Fig. 6, 
image indication assisted face recognition is preferably carried out by initially detecting 
the presence and location of an identification indication, preferably a VTVTDOT™ 
identification indication, using software commercially available from ImagelD Inc. 
[00 71.] Once the location of the identification indication is detected, the 

shape and orientation of the identification indication may be sensed to provide an 
indication of a person's orientation relative to the camera. The size of the identification 
indication provides an indication of scale. 

[00 72.] The indications of orientation and scale are employed to define a 

preferred face detection region. Typically the relative width of this region is a positive 
function of the relative width of the identification indication. The preferred face 
detection region is preferably located above the location of the identification indication 
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and is spaced therefrom, typically by a distance which is about one order of magnitude 
greater than the size of the identification indication. 

[00 73.] Face recognition is carried out within the preferred face detection 

region, preferably using any suitable face detection software, such as, for example: 

[00 74.] FACEIT R software, commercially available from Visionics 

Corporation of 5600 Rowland Road, Minnetonka, MN 55343 U.S.A. and 

[00 75.] TrueFace software, commercially available from eTrue, Inc. of 

144 Turnpike Road, Suite 100, Southboro, MA 01772 U.S.A. 

[00 76.] If face recognition is not successful with the preferred face 

detection region, it is preferably carried out also outside of the preferred face detection 
region. 

[00 77.] It will be appreciated by persons skilled in the art that the present 

invention is not limited by what has been particularly shown and described hereinabove. 
Rather the scope of the present invention includes both combinations and 
subcombinations of the various features described hereinabove as well as variations and 
modifications which would occur to persons skilled in the art upon reading the 
specification and which are not in the prior art. 
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